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Abstract 


This paper presents a methodology for evaluating the robustness of a controller 
based on its ability to satisfy the design requirements. The framework pro- 
posed is generic since it allows for high-fidelity models, arbitrary control struc- 
tures and arbitrary functional dependencies between the requirements and the 
uncertain parameters. The cornerstone of this contribution is the ability to 
bound the region of the uncertain parameter space where the degradation in 
closed-loop performance remains acceptable. The size of this bounding set, 
whose geometry can be prescribed according to deterministic or probabilistic 
uncertainty models, is a measure of robustness. The robustness metrics pro- 
posed herein are the parametric safety margin, the reliability index, the failure 
probability and upper bounds to this probability. The performance observed at 
the control verification setting, where the assumptions and approximizations 
used for control design may no longer hold, will fully determine the proposed 
control assessment. 
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Nomenclature 


C Complement set operator 

d Design parameter 

£ Feasible design space 

T Failure domain 

d Nominal design parameter 

f p (p ) Probability density function 

F p (p ) Cumulative distribution function 

Q Robust design space 

g Constraint function 

T Failure domain 

T Indicator function 

m Aspect vector of a hyper-rectangle 

Ai p Maximal set in physical space 

Ai u Maximal set in standard normal space 

n Number of samples 

p Uncertain parameter in physical space 

p Nominal parameter point in physical space 
p Critical parameter point in physical space 
P Probability operator 

1Z Hyper-rectangle 

S Hyper-sphere 

u Uncertain parameter in standard normal space 
u Nominal parameter point in standard normal space 
u Critical parameter point in standard normal space 
U Parameter transformation to standard normal space 

a Critical similitude ratio 

o; Similitude ratio 

/ 3 Reliability index 

A Support set 

Q Reference set 

ij)p Upper bound to the failure probability 

$ Cumulative distribution function of a standard normal variable 
p Parametric safety margin 
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Acronyms 

CPV 

Critical Parameter Value 

CSR 

Critical Similitude Ratio 

FORM 

First Order Reliability Method 

LQG 

Linear Quadratic Gaussian 

LTI 

Linear Time Invariant 

MS 

Maximal Set 

PID 

Proportional- integral-derivative 

PSM 

Parametric Safety Margin 

RI 

Reliability index 

SORM 

Second Order Reliability Method 
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1 Introduction 


Control verification is the last simulation-based step before control validation 
and experimental testing. As a result, a control verification setting entails 
complex, nonlinear, high-fidelity simulations where all subsystems, commonly 
developed independently having its own set of approximations, assumptions 
and methods, are integrated. Under these conditions, the vast majority of 
simplifications supporting robust and adaptive control methods — e.g., lin- 
ear dynamics, multi-affine parameter dependencies, attainment of matching 
conditions — may not hold. The methodology proposed in this paper is appli- 
cable to variable-fidelity models of the plant while permitting arbitrary func- 
tional dependencies between the stability and performance requirements and 
the uncertainty. Note that the scope of this framework precludes the usage of 
convexity conditions from the outset. 

Most of the strategies for robust control analysis are based on identifying 
the worst-case stability and performance of a Linear Time Invariant (LTI) 
system. While strategies for uncertainty in the mathematical form of the 
model, i.e., model-form uncertainty, are intrinsically conservative, those for 
real parameter uncertainties can only handle particular dependencies. Those 
strategies include affine, multi- affine [1,2] and polynomial forms [3,4], The de- 
ployment of these methods over other functional forms is not possible without 
introducing unintended conservatism. Such conservatism is highly undesirable 
in control engineering where a balance between robustness and performance is 
essential. 

For the aforementioned reasons, control verification is usually performed 
using sampling-based techniques such as Monte Carlo analysis [5-10]. How- 
ever, when accurate robustness assessments are desired, the computational 
demands associated with these methods render them impractical. This is 
especially true when there are high-consequence low-probability events of in- 
terest, e.g., the probability of instability. The foundation of this paper is the 
bounding of regions in the uncertain parameter space where the closed-loop 
performance remains acceptable. In contrast to alternative approaches, the 
method enabling the identification of such regions is optimization-based. Fur- 
thermore, the problem formulation does not introduce conservatism into the 
resulting control assessment. The size of this bounding set, whose geometry 
can be prescribed according to deterministic or probabilistic uncertainty mod- 
els, is a measure of robustness. Several of these measures are proposed herein. 
The implementation of the strategies proposed only requires a model of the 
closed-loop response that is continuously parametrized with the uncertainty, 
and a standard algorithm for constrained optimization. 

This paper is organized as follows. Basic concepts and definitions are 
introduced in Section 2. This is followed by Section 3 where the mathematical 
background required to perform set deformations according to deterministic 
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uncertainty models is discussed. Extensions enabling the usage of probabilistic 
uncertainty models are presented in Sections 4 and 5. Finally, an example and 
a few concluding remarks close the paper. 


2 Concepts and Definitions 

The object of this paper is the evaluation of the robustness characteristics of a 
closed-loop system having a parametric mathematical model. The parameters 
which specify the system are grouped into two categories: uncertain parame- 
ters, denoted by the vector p and the control design parameters denoted by 
the vector d. While the plant, actuator and sensor models depend on p, the 
controller depends on d. 

The uncertainty model of p can be deterministic or probabilistic. A de- 
terministic uncertainty model is prescribed by the Uncertainty Set A, while a 
probabilistic one is prescribed by a random vector. This vector is specified by 
the joint probability density function f p (p) defined over A. The uncertainty 
set of the probabilistic model is commonly called the Support Set. Hereafter, 
the terms uncertainty set and support set will be used interchangeably. By 
specifying the uncertainty model, it is implied that A contains the actual value 
of p, which is assumed to be time invariant. In principle, the uncertainty set 
can be unbounded and can even be the entire parameter space. In practical 
applications, the choice of this model is usually made by a discipline expert. 
Any member of the uncertainty set is called a Realization. The Nominal Pa- 
rameter point, denoted asp £ A, is an estimate of the value the uncertain 
parameter assumes under nominal operating conditions. The plant evaluated 
at the nominal parameter point will be called the Nominal Plant. Further, the 
set of control design parameters of a Baseline Controller will be denoted as d. 

Stability and performance requirements for the closed-loop system will be 
prescribed by the set of constraint functions g(p,d) < 0. The constraints 
in g may depend implicitly on time or frequency 1 . Throughout this paper, 
it is assumed that vector inequalities hold component wise. The larger the 
region in p-space where g(p,d) < 0 , the better the robustness of the baseline 
controller. In the ideal case, such a region contains the support set A. 

Sets in the parameter and design spaces, instrumental to the developments 
that follow, are introduced next. The Failure Domain corresponding to the 

1 For instance, if a requirement is that the step response y(p,d,t) must not exceed the 
upper envelope y(t), the corresponding constraint is given by 

g = ma x.{y(p, d , t) - y{t)}. 

Maximizations over t, such as this one, will be evaluated by selecting the largest value 
assumed by the argument at the discrete points of a time simulation. Hence, the evaluation 
of g will not entail solving an optimization problem per se. 
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baseline controller d is given by 2 


dim(g) 

Hd)= U (1) 

3 = 1 

where 

F j (d) = {p : g^p, d) > 0}. (2) 

The failure domain is the collection of parameter points for which the base- 
line controller violates at least one of the requirements. While Equation (1) 
defines the overall failure domain, Equation (2) defines the failure domain 
corresponding to the jth requirement. The Non-Failure Domain is the com- 
plement set of the failure domain and will be denoted 3 as C{J~). The names 
“failure domain” and “non-failure domain” are used because in the failure do- 
main at least one constraint is violated while, in the non-failure domain, all 
constraints are satisfied. The level of robustness of a controller is related to 
the size and geometry of the non-failure domain. The Feasible Design Space , 
denoted as E and defined by 


S(p) = {d:g(p,d) <0}, (3) 

is the set of designs that satisfy the requirements for the nominal plant. The 
Robust Design Space , Q, defined as 

Q(A) = {d : g(p , d) < 0, Vp e A}, (4) 


is the set of designs that satisfy the requirements for all parameter realizations 
associated with a given uncertainty model. A controller will be called Strictly 
Robust if it belongs to Q( A), i.e. , if T and A do not intersect. Otherwise, the 
controller will be called Non-Robust. Note that Q C E when p € A. 

The formulations that follow depend on whether the baseline controller sat- 
isfies the requirements for the nominal plant or not. To make this distinction, 
the function 

7 = min{fy,}, (5) 

j 


where 



if gj{p, d) < 0 
otherwise. 


( 6 ) 


is introduced. The nominal closed-loop system satisfies the requirements for 
which hj — 1 and violates those for which hj — —1. When 7 = 1, all the 
requirements are satisfied and d E E. In this case we want to determine the 


throughout this paper, super-indices are used to denote a particular vector or set while 
sub-indices refer to vector components, e.g., p] is the ?'th component of the vector p> . 

3 C(-) is the set complement operator. 
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separation between p e C{T) and the failure domain T . The larger the separa- 
tion, the better the robustness characteristics of the controller. When 7 = — 1, 
at least one of the requirements is violated. In this case we want to determine 
the separation between pef and the non-failure domain C{T). The larger 
the separation, the worse the robustness characteristics of the controller. 


3 Set Deformations in the Parameter Space 

Let fl be a set in p-space, called the Reference Set , whose geometric center is 
the nominal parameter p. The geometry of Q, will be prescribed according to 
the uncertainty in p. 

For the sake of clarity, the presentation that follows concentrates on the 
case where the baseline controller is feasible, i.e. , 7 = 1 . One of the tasks 
of interest is to assign a measure of robustness to d based on measuring how 
much the reference set can be deformed before intersecting the failure domain. 
The Homothetic Deformation of Q. with respect to p by a factor of a > 0, 
is the set H(D.,a) — {p + a(p — p) : p E 0}. The factor a, is called the 
Similitude Ratio. While expansions are accomplished when a > 1, contractions 
result when 0 < a < 1. Note that 7^(Q, 1) = ft, and 7i(0,,a 1 ) C 7i(D,,a f) 
when ol\ < a.%. Hereafter, deformations must be interpreted as homothetic 
expansions or contractions about p. 

Intuitively, one imagines that the reference set is being deformed until its 
boundary touches the failure domain, i.e., until at least one member of the 
deformed set violates one or several of the closed-loop requirements. Figure 
1 shows a sketch for a hyper- rectangular reference set. Any point where the 
deforming set touches the failure domain is a Critical Parameter Value (CPV). 
The CPV, which will be denoted as p, might not be unique. The deformed set 
is called the Maximal Set (MS) and will be denoted as A4 p . By construction, 
all the points interior to the MS satisfy the closed-loop requirements. The 
Critical Similitude Ratio (CSR), denoted as a, is the similitude ratio of the 
deformation leading to the MS. While the CSR is a non-dimensional number, 
the Parametric Safety Margin (PSM), denoted as p and defined in Section 3.3, 
is its dimensional equivalent. The values taken on by the CSR and the PSM 
are proportional to the size of the MS. 

The calculation of the CPV requires solving an standard optimization prob- 
lem 4 . Such a problem is non-convex when the dependency of g on p is non- 
linear. In any non-convex optimization problem there is always the possibility 
of convergence to the non-global optimum 5 . When this occurs, the CPV re- 

4 All the optimization problems posed in this paper were implemented using the “fmincon” 
function of the Matlab optimization toolbox. 

5 Note that the First Order Reliability Method (FORM) and the Second Order Reliability 
Method (SORM) formulations [17] widely used in reliability analysis suffer from the very 
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Figure 1. Variables in the deformation of a rectangular set for 7 = 1. The 
failure domain is colored in gray. 


suiting from the numerical optimization is mistaken and the corresponding 
MS intrudes into the failure region. This intrusion yields to a spuriously 
larger PSM. Absolute guarantees are not possible, but a variety of algorithmic 
safeguards can be used to deal with this deficiency. For instance, g can be 
evaluated at a few samples points in Ai p and if one happens to be in the 
failure domain, it can be used as an initial condition in a subsequent opti- 
mization. Requirements with an affine parameter dependency will not suffer 
from this potential drawback. Notice however, that such requirements may 
not only introduce unintended conservatism but also may misrepresent the 
desired control objective. 

Formulations that enable finding the CPV for hyper-spherical and hyper- 
rectangular reference sets are presented next. 

3.1 Deformation of Hyper-Spheres 

One possible choice for the reference set Q is the hyper-sphere 

S(p,R) = {p : ||p-p|| < R}, (7) 

where p is its center and R is its radius. This geometry enables considering 
parameters with similar levels of uncertainty. These sets could also be used 
for parameters with dissimilar levels of uncertainty if scaling is used. However, 

same deficiency. 
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whether scaling is used or not, a degree of dependence among parameters is 
introduced. The deformation of sets with this geometry can be performed as 
follows. 

The CPV for the jth requirement is given by 

p> = argmin { || p - p\\ : hjg^p, d) >0, Ap > b} . (8) 

v 

The last constraint is used to exclude regions of the parameter space where 
plants are infeasible and/or uncertainty levels are unrealistic. For instance, if 
p is composed of masses and time-delays the constraint p > 0 is implemented 
by using A — I, and 6 = 0. 

For 7=1, the overall CPV is 


p=p\ (9) 

where 

k = argmin {Up 7 — p||}. (10) 

Hence, the CPV for each individual requirement is identified and the closest of 
these CPVs to the nominal parameter point is the overall CPV. Notice that the 
critical requirement, i.e., any requirement that prevents a larger deformation, 
is g k < 0. 

For 7 = — 1, the overall CPV is given by 

p = argmin {||p — p|| : g(p , d ) < 0, Ap > 6} . (11) 

v 


In this case, the value of p may not coincide with any of the p's. This is 
because the overall CPV is at the intersection of the individual non-failure 
domains. Furthermore, if there exists a j such that p 7 does not exist, meaning 
that the deformation grew unbounded, the overall CPV does not exist either. 
Note that for 7 = 1, the CPV is the p point in the failure domain that is the 
closest to p, while for 7 = —1 the CPV is the p point in the non-failure domain 
that is the closest to p. Figures 2 and 3 illustrate the 7 = 1 and 7 = — 1 cases 
respectively. The failure domain, which is the union of both individual failure 
domains, is colored with gray. 

The MS and the CSR are uniquely determined by the CPV according to 


— 
a — 


S(p, aR ), 

IIp-pII 

R ' 


( 12 ) 

(13) 


Note that A 4 P is the largest deformation of Q which fits within C(JF) for 7 = 1 
and within T otherwise. The MS in figures 2 and 3 is shown as a blue circle. 
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Figure 2 . Relevant variables in the 7 = 1 case. The failure domain and the 
maximal set are colored in gray and blue respectively. 



Figure 3. Relevant variables in the 7 = — 1 case. The failure domain and the 
maximal set are colored in gray and blue respectively. 


In the particular case when dim{p} — 1 , the MS is the interval A4 P — (p — 
p,p+p), where p — \p—p\. Since this case only considers uncertainty in a single 
parameter, the resulting robustness metrics cannot capture the effects of the 
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interaction among such parameters on the requirements. Therefore, the CPV 
of a n dimensional deformation might be much closer to the nominal parameter 
point than the n one-dimensional CP Vs resulting from deforming each of such 
parameters individually. The sketch in Figure 4 illustrates this setting for the 
design requirements g 1 < 0 and g 2 < 0. Note that the non-failure domain 
is the intersection of the individual non-failure domains. Besides, the overall 
CPV is the parameter point closest to the nominal point where at least one 
component of g is equal to zero. Since in the case shown this component is 
g 2 , k = 2 and p = p 2 . Therefore, g 2 < 0 is the critical requirement. Note that 
A i p , the MS corresponding to the overall CPV for which both constraints are 
active, is the intersection of the MSs associated with each individual constraint. 

3.2 Deformation of Hyper- Rectangles 

Another possible choice for the reference set D is the hyper-rectangle 

TZ(p, m) — {p: p — m<p<p + m} , (14) 

where m > 0 is the vector of half-lengths of the sides. Rectangular sets permit 
the consideration of parameters with dissimilar levels of uncertainty having no 
dependence. Such levels are attained by making m* proportional to the level 
of uncertainty in 

Denote by ||a?||“ = maxj{|ccj|/mj}, the m-scaled infinity norm. A distance 
between the vectors x and y can be defined as ||x— y||^. Note that 7 Z(p, m) — 
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{p : \\p — p||~ < 1}. Formulations that enable deforming sets with this 
geometry are presented next. 

The CPV in the hyper-rectangular case is given by Equations (8-11) after 
replacing the Euclidean norm with the m-scaled infinity norm. The “max” 
resulting from this substitution can be eliminated by introducing the similitude 
ratio a defined earlier. This leads to the following expression for the jth 
individual CPV 

(p 7 , oP) — argmin {a : hjg^p, d) > 0, Ap > b, (15) 

p,a 

p — am <p<p + am} . 

In this setting, the overall CPV is the individual CPV that is the closest to 
the nominal parameter point in the sense of the m-scaled infinity norm. 

The MS and the CSR are 


M p - 7 Zip, am), (16) 

a = l|p-p|lm- ( 17 ) 

3.3 Robustness Metrics for Deterministic Uncertainty 
Models 

Because the CSR measures the size of the MS, its value is proportional to the 
degree of robustness of the controller d to uncertainty in p. The CSR is a 
non-dimensional metric whose value depends on the size of the reference set. 
The PSM, which is the dimensional equivalent of the CSR, serves the same 
purpose without having this dependency. The Spherical PSM is 

Ps(P ) = 1 &R, (18) 

where the CSR is given by Equation (13), while the Rectangular PSM is 

Pr(P) =7«IW> ( 19 ) 

where the CSR is given by Equation (17). The sign convention enforced by 7 in 
these equations implies the following: if the PSM takes on a negative value, the 
controller does not even satisfy the requirements for the nominal plant. If the 
PSM is zero, the controller exhibits no robustness because there are arbitrarily 
small perturbations of p from p leading to a constraint/requirement violation. 
If the PSM is positive, the requirements are satisfied by the nominal plant and 
those around it. The larger the PSM, the larger the fl-shaped neighborhood 
of p for which the requirements are satisfied. 

The PSMs corresponding to individual requirements result from using oP 
and hj, instead of a and 7 , in Equations (18-19). This yields to 

Ps(p0 = hjWpi -p||, (20) 
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Pr{&) = hjWp’ -pOm||. (21) 

The individual PSMs enable comparing the levels of robustness associated 
with each requirement. When 7 = 1 , the smallest individual PSM is equal 
to the overall PSM, i.e., p{p) — mm-fyfy* 7 )}. When 7 = — 1 however, the 
smallest individual PSM is just an upper bound to the overall PSM, i.e., p(p) < 
mufyp^)}. The trade-off between conflicting requirements will be reflected 
by the values taken on by the individual PSMs. Controllers that maximize the 
overall PSM will incidentally attain a trade-off that optimally balances the 
robustness for all requirements. 

Traditionally, robust control analysis is made by setting forth an uncer- 
tainty set and checking if the stability and performance for all set members 
is satisfactory. This applies to methods for both, real parameter and model 
form uncertainty. Determination of whether a controller is robust or not for a 
given hyper-spherical or hyper-rectangular set A, is a matter of making Q — A 
and calculating the CSR. The controller is strictly robust if and only if the 
CSR is greater or equal to one. A controller might turn out to be non-robust 
because of an overly-large uncertainty set or because the requirements are too 
stringent. If the size of uncertainty model is not reduced, and the design re- 
quirements are not relaxed, the only choice is to adopt more complex control 
structures until a robust controller is found. This practice however, usually 
lead to high-order controllers. Alternatively, one may enable the usage of sim- 
pler control structures by allowing for requirements violations with a small 
probability. Strategies based on this idea are presented next. 

4 Probabilistic Uncertainty Models 

Hereafter, it is assumed that a probabilistic uncertainty model of p is available. 
Recall that such a model is prescribed by a random vector having the joint 
probability density function f p (p), defined over the support set A. Note that 
the only information from such a model required for determining whether 
a controller is strictly robust or not is its support set. The developments 
that follow are suited for controllers that are non-robust. The analysis of 
such controllers entails quantifying the severity by which the requirements are 
violated. In the context of deterministic uncertainty models, one metric for 
this is the volume of A fl C(M P ). 

Probabilistic uncertainty models enable the discrimination among all pos- 
sible parameter realizations. The value of the joint density function at each 
particular realization can be interpreted as a measure of our belief that such 
a realization is the actual value of the uncertain parameter. In this setting, 
a natural metric of robustness is the probability of violating the design re- 
quirements. This probability, called the Failure Probability , will be denoted as 
The failure probability is zero for robust controllers, while it is greater 
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than zero for non-robust ones. Clearly, the smaller P[P], the more robust is 
the controller. Note that the set A fl C(M P ) above is an approximation of 
the failure domain and its volume is the failure probability for the case when 
uncertainties are uniformly distributed. It is worth noticing that a controller 
may attain a small value of P[J-\ but fail to satisfy the requirements for the 
nominal plant. 

The developments that follow enable the comparison of controllers with 
varying levels of complexity from a practical perspective. For instance, if a 
simple Proportional-Integral-Derivative (PID) controller satisfies the closed- 
loop requirements for 99% of the plants, it is worth determining how much 
complexity are we willing to accept for that extra 1%. One may accept using 
a simpler, non-robust controller when the corresponding probability of failure 
is sufficiently small. The pursuit of zero failure probability usually leads to 
complex controllers whose robustness may be very close to that of much simpler 
ones for most of the plants. 

In contrast to the contributions in [11-14] on probabilistic controls, the 
developments herein are not intended to alleviate the computational burden of 
pursuing robust stability, but to enable the discrimination among events in the 
set of all possible closed-loop system performances according to their chance of 
occurrence. This has the potential to address and alleviate the demands on the 
controller complexity imposed by worst-case control policies. In this regard, 
this paper is better aligned with the developments in References [8-10] and 
with the idea of handling some of the requirements as soft constraints [15, 16]. 


5 Set Deformations in Standard Normal Space 

The calculation of probabilities can be better performed in spaces other than 
p-space. The most relevant of these spaces is the standard normal space, com- 
monly referred to as the u-space. In u-space, the multi- variate probability 
density function is an uncorrelated normal density function with zero mean 
and unit standard deviation, whose value decreases as exp(— ||u|| 2 /2). For most 
probability distributions, there is a probability preserving parameter transfor- 
mation [17], denoted hereafter as u — U(p), that maps the requirements from 
p-space to u-space such that P\U(J-)\ — P[P\- 

The formulations introduced in Section 3 that enable the deformation of 
sets in p- space can be easily extended to the u-space. In this setting, the sets 
<S(u, R) and 7 Z(u, m) will be used as reference sets. A natural choice is u = 0 
because in the standard normal space most of the probability is concentrated 
about the origin. An obvious parallelism between the concepts, notions and 
equations introduced earlier and the ones in u-space is apparent. For instance, 
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if Cl — S(u , R) the CPV corresponding to the jth requirement is given by 

v? — argmin {||u — u|| : hjgj(JJ~ l (u), d) > 0} , (22) 

U 

where hj results from using Equation (6) with p — U~ l (u). Note that this 
equation is analogous to Equation (8). The constraint Ap > b is no longer 
used since its intent can now be achieved by using a suitable A in f p (p). Note 
that if the baseline controller is robust, U (J 7 ) is empty and all the optimization 
problems posed in this section have no feasible solution. 

The formulation for the overall CP Vs can be easily inferred from the above 
developments, e.g., for 7 = 1 the overall CPV is u — u k , where k results 
from using \\vP — it|| in Equation (10). The MS and the CSR corresponding 
to hyper-spherical sets are 


M. u = S(u, aR), 

(23) 

1 ? 

1 ft; 

11 

>8 

(24) 

while those for hyper-rectangular sets are 


M u = 7 Z(u, am), 

(25) 


(26) 


Note that M u is the largest deformation of Cl which fits within U(C(J-)) for 
7 = 1 and within U (J 7 ) otherwise. 


5.1 Robustness Metrics for Probabilistic Uncertainty 
Models 

Metrics based on probabilistic uncertainty models and methods for their cal- 
culation are introduced next. 

5.1.1 Reliability Indices 

The Reliability Index (RI) is the analogous to the PSM in ■u-space. In partic- 
ular, the Spherical RI is defined as 

p s (u) = 7 &R, (27) 

where the CSR is given by Equation (24), while the Rectangular RI is 

Pr(u) = 7«IMI, ( 28 ) 

where the CSR is given by Equation (26). The RIs corresponding to individual 
requirements are 

/3s(u j ) = hj\\u j — u||, (29) 
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PR(u j ) = hj\\u j - «||~||m||. (30) 

As with the PSMs, /3 ( u ) = min{/?(ii J ’)} for 7 = 1 and /3(u) < min{/?(iP)} for 
7 = -1. 

In p-defor mat ions, all the information on the relative levels of uncertainty 
in the components of p is determined by the geometry of the reference set. 
In n-defor mat ions, this information is prescribed by both the reference set 
and the probability density function. To preserve the relative levels of uncer- 
tainty of the probabilistic model, the reference set should be a hyper-sphere 
or a hyper-cube centered at the origin. Note however, that the geometry of 
the reference set can be adjusted as desired. The developments in the next 
section indicate that reference sets leading to large maximal sets are desirable. 
Such sets may not preserve the uncertainty levels implied by / P (p). Strategies 
for the calculation of upper bounds to the failure probability based on the 
deformations of sets in p- and u-space are presented next. 


5.1.2 Bounds to Failure Probability 

Let us define ip = P[C(M)\ = 1 — P[Ai\, where At is a MS in either p- or 
u-space. Note that P\P\ < ip for 7=1 since T C C(M). Furthermore, 
P[J-\ > 1 — ip for 7 = — 1 since At C T . For the reminder of this section our 
discussion concentrates on the case where 7 = 1 so ip is an upper bound to 

p\n 

For deformations in p-space we will use the notation 


ip P = 1 - P[M p \. (31) 

If the components of p are independent random variables and Vt = 7 Z(p, m), 
we obtain 


dim(p) 

ip p (p, m, pr) = 1 — Yl 

i= 1 





. (32) 


where F p is the cumulative distribution function and pr is given by Equation 
(19). Upper bounds to P [A 7 ] can be calculated by using the individual PSMs 
from Equation (21) into Equation (32). Since A 4 P and therefore pr, are inde- 
pendent of the uncertainty model f p , upper bounds corresponding to different 
uncertainty models can be calculated with minimal computational effort. 

Tighter bounds are obtained if the reference set chosen leads to a more 
probable MS. This can be easily attained by working in the standard normal 
space. For deformations in u-space we will use the notation 


ip u = 1 - P[M U \. (33) 

While ip p can be estimated analytically for hyper-rectangular sets, ip u can 
be estimated analytically for hyper-spherical sets centered at u — 0 and for 
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hyper-rectangles. The following Lemma 6 enables the calculation of for the 
former case. 


Lemma 1. If Ui for i — 1,2 , ,1 is a set of independent standard normal 
variables, A{l,r ) = P[||n|| < r] is given by 

erf (a) ~ e ~ v \/i ES (Syrr if i ^ odd 

1 - e-V £ ? =1 (Sji! */ 1 is even 


A(l,r) 


(34) 


where !! is the double factorial operator 7 . 

In the case where ft — S(u, R) and u = 0 , Equations (33-34) lead to 


'ipu(Ps) = 1 - A(dim(p), |/3 5 |). (36) 

Upper bounds to P [P- 7 ] can be calculated by using the individual RIs from 
Equation (29) into Equation (36). 

In the case where ft = lZ(u, m), we have 

dim(p) 

ip u (u, m,0 R ) = l- 

1=1 

where 4> is the cumulative distribution function of a standard normal random 
variable and (3r is given by Equation (28). Upper bounds to P\T^] can be 
calculated by using the individual RIs from Equation (30) into Equation (37). 
Because of the parameter transformation u — U(p), the calculation of ip u for 
a new uncertainty model f p requires solving for a new Ai u and therefore for 
a new RI. This is in contrast to the bound in Equation (32) where a change 
in f p can be readily accommodated for. 

Because the bounds result from calculating the probability of an event their 
value lies between zero and one, i.e., ip p E (0, 1] and fj u & (0, 1]. For a given 
f p , g and d, there is no general rule that determines which reference set leads 
to the smallest upper bound. Equations (32), (36) and (37) are guaranteed 
upper bounds to the failure probability and as such they do not suffer from 
approximation error. Note that the efficiency of the methods used to calculate 
the PSMs, the RIs and the bounds is independent of the value of P[P\- 

6 The corresponding proof is available in reference [18]. 

7 The double factorial [19] is defined as 


1 

f n- 

(n -2)- 

• • 5 ■ 3 • 1 

n > 0 and odd 

n\\ = 

?■ 

(n — 2) • 

■ ■ 6 • 4 ■ 2 

n > 0 and even 
n = —1, 0 


4 > [Ui + 


\mrni 

llmll 


$ [ Ui 


\PR\mi 

llmll 


. (37) 
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5.1.3 Failure Probability 

The problem of finding P[J-], or equivalently P[U(P)], is usually difficult since 
it requires evaluating a multi-dimensional integral over a complex integration 
domain. Methods for approximating the failure probability include sampling 
based techniques, FORM [17], SORM and the hybrid method [18,20]. An 
outline of the latter approach follows. 

The hybrid method approximates the failure probability by combining sam- 
pling with the bounds derived above. Once the MS has been determined, an 
arbitrary sample point of f v can be easily tested for membership in this set. 
If the sample point lies within the MS, it is unnecessary to go through the 
computational expense of evaluating g , since the outcome is now known. The 
numerical advantage of using this method is a consequence of not having to 
evaluate g at these realizations. In particular, 

( 38) 

z n 

i=i 

where ip is given by Equation (32), (36) or (37), p % for i — 1, 2, ... n are samples 
of f p in C(A4), and X is the indicator function 

2e^e. (39) 

An approximation to P \P\ results from using the variables corresponding to 
the jth requirement. For instance, if Q = TZ(p, ra), we have 

P [JP] w (1 ~ Ml ~ tA + ^Tx(p\ P), (40) 

Z Tl 

i=l 

where 'ipp results from substituting the pn in Equation (21) into Equation (32) 
and every p l falls into C(lZ(p , \\fP — p||“m)). Note that Equation (1) im- 
plies that P[P] > maxjPfJ 7 - 7 ]}. More efficient implementations of the hybrid 
method are available [18,20]. 

This method is especially advantageous when the number of evaluations of 
g required to determine the MS is considerably less than the number of sam- 
ples falling within the MS. This is usually the case when P[T] 1. While the 
efficiency of the method increases with the CSR, its accuracy is comparable to 
the one of Monte Carlo using floor (n/ip) samples [20]. A particularly attractive 
feature of the hybrid method is that its efficiency and accuracy does not de- 
pend on the robustness of the controller [20] . This sharply contrasts with the 
case in sampling-based methods, where accurate robustness assessments (i.e., 
those requiring statistics having small confidence intervals) demand a number 
of samples that grow exponentially fast as P[P] approaches zero. Figure 5 
illustrates the dependency of the Monte Carlo approximation Pf to P[P] on 
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the minimum number of samples required to attain an upper limit of the 95% 
confidence interval equal to 2 Pf. Note that accurate approximations to small 
failure probabilities require an exceedingly high number of g evaluations. 



Monte Carlo approximation to P[F] 


Figure 5. Number of samples required to accurately approximate failure prob- 
abilities via Monte Carlo sampling. 


While the RIs and the upper bounds to the failure probability have a no- 
tion of robustness based on the closeness of the nominal parameter point to 
the failure domain, the failure probability does not. As such, the estimation 
of P[J-\ does not require the definition of a nominal parameter point and de- 
signs attaining small failure probabilities may have their corresponding failure 
domain close to regions where f p (p) is large. While a small P[J-] is desirable, 
only a sizable value of the RI guarantees that the nominal parameter point is 
separated from T . While the reduction of P[P] does not necessarily increase 
this separation, an increase of the RI not only augments the separation but 
also reduces P[P\- 


6 Example 

The comparative analysis of controllers designed for the robust control chal- 
lenge problem posed in the 1990 American Control Conference [21] is presented 
next. This problem was chosen because the public availability of the controllers 
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enable the reproducibility of the results. Note however, that the methodology 
developed is applicable to far more complex problems than the one herein. 

The benchmark system, shown in Figure 6, is a two- mass/spring system 
with a non-collocated sensor actuator pair. Several design problems were posed 



Figure 6. Two-mass spring system. 


based on this setting. As in Reference [22], additional uncertainties are con- 
sidered to fully exercise the scope of the methodology. We added a non-linear 
spring with constant k n , a time delay r denoting a first-order lag between con- 
troller command and actuator response and a control effectiveness uncertainty 
/ resulting from variation in sensors, control gain and actuator failure. Hence, 
P = [mi, m 2 , k,k n ,T, f] T . The plant model is 


Xl = x 3 
X 2 = X 4 

. k / . k n , . q f xl 

x 3 — — (x 2 — Xi) -f (x 2 - xi) + — , 

mi mi mi 

x 4 = — (xi — x 2 ) + — (xi — X 2 ) 3 + — , 

m2 m2 m2 

TU — U c — U. 


Stability and performance requirements, prescribed as 


1. Local closed-loop stability. 

2. Settling time: the output to a unit-impulse disturbance w must fall 
between ±0.1 after 15s. 

3. Control saturation: the control for a unit-impulse disturbance must fall 
between ±1. 


lead to 


9 - 


max {!ft(s*)}, max{|x 2 |} — 0.1, 


*>15 


T 


max{|'u|| — 1 
*>o 11 IJ 


where s' 1 is a closed- loop pole of the linearized system and -ft(-) is the real 
part operator. Eleven controllers designed by several authors using different 
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Controller 

Gain margin 
db 

Phase margin 

deg 

PR (P 1 ) 
Stability 

PR (P 2 ) 
Settling time 

PR (P 3 ) 
Saturation 

PR (P) 
Overall 

A 

2.5 

-20.6 

0.320 

-0.019 

0.368 

less than zero 

B 

3.2 

26.7 

0.490 

-0.175 

0.523 

less than zero 

C 

3.3 

26.6 

0.501 

-0.189 

0.533 

less than zero 

D 

15.1 

57.2 

0.562 

0.212 

—00 

less than zero 

E 

2.4 

22.2 

0.394 

-0.259 

0.233 

less than zero 

F 

5.2 

23.9 

0.419 

0.012 

-3.277 

less than zero 

H 

3.4 

24.8 

0.510 

-0.005 

0.524 

less than zero 

Wi 

3.3 

24.5 

0.501 

0.001 

0.517 

0.001 

W 2 

6.0 

34.2 

0.754 

-0.024 

0.639 

less than zero 

Bi 

1.8 

18.9 

0.300 

0.020 

0.003 

0.003 

b 2 

2.8 

27.2 

0.437 

0.038 

0.141 

0.038 


Table 1. Stability margins and PSMs. 


methods are studied here. These controllers will be labeled as A, B, C , D, E, 
F and H (from [22] and references therein); W\ and W 2 (from [23]); and B\ 
and B 2 (from [24]). The structure of the controllers is as follows: controllers A , 
B, and C used loop transfer recovery; controllers D, H, W\ and W 2 are based 
on //-infinity control; controller E is designed by optimization; controller F 
is Linear Quadratic Gaussian (LQG) based; and controllers Bi and B 2 were 
designed using //-synthesis. 

The deformation of the reference set Cl = 1 Z(p, m) is considered first. We 
assume that p = [1, 1, 1, 0, 0, 1] T and m = [4, 4, 5, 2, 1, 2] T , e.g., we expect four 
times more uncertainty in the value of the masses than in the value of the time 
delay. The pair (p, m) , which constitutes a deterministic uncertainty model, 
is, as any mathematical model of uncertainty, subjective. Notice however, 
that the MS for 7 = 1 is fully contained in the non-failure domain regardless 
of the value this pair takes on. In order to prevent deformations leading to 
infeasible plants, the constraints mi > 0, m 2 > 0, k > 0, r > 0 and / > 0 
in the form of Ap > b were added. Table 1 presents the stability margins 
for the nominal plant and the PSMs for the three requirements. Results on 
Table 1 show that only W \ , Bi and B 2 satisfy the requirements for the nominal 
plant (i.e., the PSMs are positive). While W 2 has the best figure of merit in 
regard to stability and control saturation, the settling time requirement is best 
satisfied by D. Note that the stability margins are not reliable indicators of 
robust stability. For instance, while D has the best stability margins, it does 
not attain the largest pn (p 1 ) . According to the PSM, B 2 is the best controller 
since it attains the largest Pr(p), which is 0.038. 

Control assessments for a probabilistic uncertainty model are presented 
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Controller 

Ps (u l ) 

Stability 

fe (u 2 ) 

Settling time 

Ps (« 3 ) 

Saturation 

few 

Overall 

A 

0.665 

-0.037 

0.913 

less than zero 

B 

0.992 

-0.319 

1.169 

less than zero 

C 

1.01 

-0.336 

1.191 

less than zero 

D 

2.366 

0.598 

—00 

less than zero 

E 

0.690 

-3.517 

0.374 

less than zero 

F 

1.627 

0.025 

—00 

less than zero 

H 

1.050 

-0.009 

1.174 

less than zero 

Wi 

1.027 

0.0009 

1.152 

0.0009 

W 2 

2.147 

-0.072 

2.287 

less than zero 

Bi 

0.497 

0.030 

0.005 

0.005 

b 2 

0.852 

0.066 

0.236 

0.066 


Table 2. Spherical RIs. 


next. We assume that mi, m 2 , k, k n , r and / are independent, Beta-distributed 
random variables with shape parameters, [5, 5], [5, 5], [2, 3.7], [6, 6], [0.3, 5] and 
[1,1] and support sets [0,2], [0,2], [0.5,2], [—0.5, 0.5], [0,0.1] and [0.5, 1.5], 
respectively. The value of h corresponding to u — 0 makes the signs of the 
individual PSMs and RIs to coincide. 

The deformation of Q — S(u, R) for u — 0 leads to the metrics shown in 
Table 6. According to the RIs, the controller D is the one with best stability 
and settling time characteristics while W 2 has the best figure of merit for 
control saturation. According to the RI, B 2 is the best controller since it 
attains the largest /?s(u), which is 0.066. The bounds to the failure probability 
can be readily calculated from the entries of Table 6 and Equation (36). The 
control assessment resulting from these bounds coincide with the one based 
on the RIs. 

The values of P[.F J ], calculated using the hybrid method for n = 1000 
samples and the RIs in Table 6, are shown in Table 6. Note that D has the 
best robustness in regard to the stability and settling time requirements, while 
W 2 is the best for control saturation. The values of P[J-] for Wi, B\ and B 2 are 
0.908, 0.980 and 0.873 respectively. Recall that T is the union of all individual 
failure domains, and as such it cannot be calculated from the data in Table 6. 
According to the failure probability, B 2 is the best controller since it attains 
the smallest PfJ 7 ], which is 0.873. Since the failure probabilities are large, the 
advantage of using the hybrid method is moderate. 

Note that all the figures of merit provide consistent robustness assessments. 
The individual PSMs and RIs indicate not only which is the limiting design 
requirement, but more importantly, how the levels of robustness corresponding 
to all requirements compare among themselves. 
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Controller 

p^ 1 ] 

Stability 

r[r*\ 

Sett, time 

p[p 3 ] 

Saturation 

A 

0.272 

0.971 

0.158 

B 

0.143 

0.957 

0.119 

C 

0.130 

0.967 

0.104 

D 

0.007 

0.333 

1 

E 

0.219 

0.999 

0.426 

F 

0.080 

0.871 

1 

H 

0.138 

0.914 

0.188 

Wi 

0.119 

0.907 

0.174 

W 2 

0.014 

0.875 

0.010 

Bi 

0.313 

0.976 

0.601 

b 2 

0.196 

0.798 

0.462 


Table 3. Failure probabilities. 


7 Concluding Remarks 


This paper proposes a computational framework for evaluating the robustness 
of a controller based on its ability to satisfy the design requirements. The 
framework developed is applicable to linear and non-linear systems subject to 
stability and performance requirements having an arbitrary functional depen- 
dency on the uncertainty. These metrics are the parametric safety margin, 
the reliability index, the failure probability and upper bounds to this proba- 
bility. The methods used to calculate these metrics are based on the solution 
of standard optimization problems. The parametric safety margin evaluates 
the robustness characteristics of the controller according to a deterministic un- 
certainty model. The other metrics, which require a probabilistic uncertainty 
model, account for the chance of occurrence of any given plant within the 
uncertainty set and therefore for the likelihood of unsatisfactory closed-loop 
performance. As compared to sampling-based methods, the efficiency of the 
tools used to calculate the proposed metrics is independent of the value they 
take on. This is particularly advantageous when the controllers are highly 
robust, a case when the computational expense of an accurate sampling-based 
analysis is impractical. The proposed methodology enables a fair comparison 
of controllers designed using different methods and assumptions. The justifi- 
cation for adopting complex control architectures and the determination of the 
consequences of violating such assumptions are aspects that can be naturally 
evaluated using this framework. 
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